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ABSTRACT 


This  paper  studies  the  value  of 
aircraft  simulators  as  measures  of 
training  readiness.  Simulator  evalua¬ 
tions  are  analyzed  for  reserve  enlisted 
crewmen  on  Navy  patrol  aircraft.  Part- 
time  reservists  are  found  to  have  very 
little  skill  loss  over  time  and  perform 
as  well  as  their  full-time  counterparts. 
Experience  in  the  simulator  produces 
substantial  increases  in  subsequent 
evaluation  scores.  Simulated  flights 
appear  to  be  useful  measures  of  readi¬ 
ness  and  valuable  training  experience. 
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INTRODUCTION 


To  what  extent  is  the  performance  in  the  military  of  an  individual 
explained  by  his  personal  characteristics,  such  as  intelligence  or 
education,  or  by  the  training  he  obtains  in  the  service?  The  empirical 
investigation  of  these  issues  is  an  area  of  manpower  research  that  has 
been  underdeveloped  in  the  past  but  in  which  some  progress  is  being 
made.  First,  this  paper  briefly  overviews  the  areas  of  research  of 
interest  as  well  as  the  types  of  approaches  utilized  in  previous 
analyses.  Then  it  turns  to  an  empirical  analysis  of  operational 
readiness  and  training. 

BACKGROUND 

Measuring  the  performance  of  military  personnel  is  an  important 
issue,  but  it  is  one  that  has  proved  to  be  extremely  difficult. 
Improvements  In  manning  levels  during  the  last  several  years  have  coin¬ 
cided  with  increases  in  reported  readiness  throughout  the  military. 

While  these  trends  do  indicate  that  personnel  and  training  policies  do 
have  an  impact  on  the  ability  of  the  military  services  to  perform  their 
missions,  manpower  analysts  are  still  unable  to  accurately  measure  the 
impact  of  specific  factors  on  the  performance  of  individuals  In  the 
military.  Similar  measurement  difficulties  impede  our  ability  to  assess 
the  impact  of  individual  performance  on  the  collective  effectiveness  of 
larger  units  of  personnel. 

Substantial  effort  has  been  devoted  to  the  issue  of  measuring  the 
performance  of  new  recruits  in  the  military.  This  work  has  been  divided 
Into  two  basic  areas  that  can  be  characterized  as  measures  of  surviv¬ 
ability  and  trainabili ty .  Survivability  is  measured  by  successful 
completion  of  a  recruit* s  obligated  service.  The  dominant  predictor  of 
survival,  as  confirmed  by  numerous  studies,  is  education  [1].  In  parti¬ 
cular,  having  obtained  a  high  school  diploma  is  the  best  predictor  of 
survival  for  enlisted  personnel.  While  survival  is  an  important  measure 
of  effectiveness,  indeed  it  is  a  prerequisite  for  any  contribution  to 
the  military,  it  does  not  provide  any  measure  of  the  relative  contribu¬ 
tion  of  those  personnel  who  do  remain  in  the  military. 

Trainability  measures  the  successful  completion  of  the  classroom 
training  provided  to  incoming  recruits.  School  pass  rates  or  final 
examination  scores  are  more  refined  measures  of  the  performance  of 
military  recruits  than  survival  rates.  Scores  on  the  military  entrance 
exam  are  the  best  predictors  of  training  success  for  new  recruits. 

While  training  success  is  a  valid  measure  of  recruit  performance,  it  is 
only  a  rough  proxy  for  subsequent  performance  in  the  military. 

Other,  more  direct,  measures  of  performance  have  also  been  anal¬ 
yzed.  These  include  supervisory  ratings  of  recruits  and  advancement 
rates  for  new  recruits  [2,  3].  While  these  measures  have  more  analytic 
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content  than  previous  measures,  there  is  still  an  insufficient  body  of 
research  to  make  these  types  of  efforts  influential  in  policy  decisions. 

Research  on  career  personnel  has  focused  less  on  the  individual  and 
more  on  the  relationship  of  the  mix  of  skills  in  a  unit  to  its  effec¬ 
tiveness.  These  types  of  studies  have  tended  to  use  conventional 
measures  of  readiness  that  focus  on  material  condition  of  the  unit  [4]. 
While  these  studies  have  frequently  found  significant  relationships 
between  personnel  characteristics  and  readiness,  they  are  frequently  the 
subject  of  attack  because  they  are  measuring  something  other  than  war¬ 
fighting  capability.  There  have  been  few  studies  that  have  tried  to 
directly  measure  operational  performance  [5]. 

OPERATIONAL  READINESS  AND  TRAINING:  EMPIRICAL  ANALYSIS 

This  study  concentrates  on  three  areas  that  have  not  been  the 
subject  of  much  research  in  the  past.  First,  operational  data  that  are 
realistic  measures  of  wartime  capability  are  analyzed.  Second,  in 
addition  to  estimating  the  effect  of  factors  such  as  military  experience 
and  personnel  characteristics  on  performance,  it  closely  examines  the 
effect  of  mid-career  training  on  the  performance  of  personnel.  In 
particular,  this  study  looks  at  the  impact  of  training  in  flight  simu¬ 
lators  on  crew  performance.1  Finally,  this  study  looks  at  a  segment  of 
the  military  that  has  not  been  a  subject  of  intense  interest  in  the 
past,  the  reserve  component  of  the  force.  The  reserves  serve  for  only  a 
limited  amount  of  time  each  year  and  this  leads  to  significant  questions 
about  their  readiness  and  policies  that  can  limit  skill  loss  as  their 
time  away  from  active  duty  increases. 

Basic  Data 


The  study  team  gathered  simulator  exercise  results  for  a  sample  of 
nine  Naval  Selected  Reserve  P-3  squadrons  from  1980  through  1982.  The 
P-3  is  a  long-range  patrol  aircraft  whose  primary  mission  is  antisub¬ 
marine  warfare.  Grew  members  on  ASW  aircraft,  both  officers  and 
enlisted,  spend  a  substantial  amount  of  time  in  simulators.  These 
sessions,  which  often  last  several  hours,  are  used  both  as  training 
exercises  and  as  indicators  of  crew  readiness.  The  individual  crew 
members,  and  the  crew  as  a  whole,  are  graded  on  their  performance,  and 
results  of  these  exercises  are  recorded.  The  sample  includes  365 
simulator  trials  and  over  1,000  individual  exercise  grades. 

In  addition  to  simulator  flights,  actual  operational  flights  are 
graded  as  well.  Data  on  grades  from  these  flights  were  collected. 
Unfortunately,  these  data  are  too  limited  to  be  useful  in  these  analy¬ 
ses.  Reserve  units  fly  a  small  number  of  operational  missions  each  year 


1.  The  term  "flight  simulator"  is  used  generically.  The  equipment  used 
by  the  crew  members  is  more  accurately  termed  a  Weapon  Systems  Trainer. 
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and  tiie  data  set  derived  from  operational  records  was  too  small  to  be 
used.  This  problem  is  exacerbated  by  the  fact  that  these  operational 
flights  tend  to  be  concentrated  among  a  small  number  of  crews  within  the 
squadron.  As  a  result,  only  information  from  simulated  flights  are  used 
in  this  study. 

This  study  concentrates  on  the  three  enlisted  sonar  operators  in 
the  aircrew.  Individual  exercise  grades  were  merged  with  personnel 
files  that  included  extensive  information  on  individual  characteristics 
and  on  Navy  training  and  experience.  The  information  contained  on  these 
records  can  be  categorized  into  four  areas:  personal  characteristics, 
flight  characteristics,  reserve  training  information,  and  simulator 
experience. 

The  Model 


Three  factors  fall  into  the  .category  of  personal  characteristics. 
The  first  is  the  paygrade  of  the  individual.  An  individual's  paygrade 
is  a  measure  of  his  experience  in  the  Navy  as  well  as  his  advancement 
rate.  In  previous  studies  of  this  type,  this  Is  the  most  powerful 
predictor  of  an  individual's  performance.  The  second  measure  is  the 
AFQT  score  earned  by  the  individual  on  his  military  entrance  examin¬ 
ation.  AFQT  is  a  composite  score  from  several  tests  on  the  Armed 
Services  Vocational  Aptitude  Battery.  AFQT  measures  verbal  and 
arithmetic  aptitude  and  is  normally  thought  of  as  a  proxy  for 
intelligence.  The  final  measurable  characteristic  is  the  educational 
level  of  the  crewman.  In  the  data  set  for  this  study,  many  of  the 
reservists  have  completed  additional  years  of  schooling  after  leaving 
the  Navy,  so  this  variable  does  not  measure  their  education  at  the  time 
they  entered  the  service. 

The  score  an  individual  receives  on  his  flight  evaluation  depends 
not  only  on  his  individual  proficiency  but  also  on  the  characteristics 
of  the  job  he  must  perform.  Certain  specific  tasks  he  must  perform  may 
be  more  or  less  difficult  from  others.  There  are  three  sensor  positions 
on  the  aircraft.  Sensor  1  and  Sensor  2  operate  the  acoustic  detection 
devices.  Sensor  1  Is  the  lead  operator  with  Sensor  2  providing 
support.  Sensor  3  operates  the  nonacoustic  devices  and  has  a  limited 
role  on  many  flights.  The  analysis  In  the  paper  takes  account  of  the 
Individual's  position  on  each  flight.  The  evaluation  scores  also  depend 
on  the  type  of  mission  being  simulated.  The  mission  type  is  a  factor  in 
the  analyses  that  follow. 

The  focus  In  this  paper  is  on  the  reserve  component  of  the  total 
force.  Information  that  pertains  directly  to  the  reserves  is  included 
in  the  analysis.  A  key  concern  in  any  analysis  of  reserve  readiness  is 
a  comparison  of  the  performance  of  reservists  to  active  duty  personnel. 
As  noted  above,  there  are  not  sufficient  data  to  compare  reserve  to 
active  personnel.  Within  the  reserve  community,  however,  there  are  two 
types  of  personnel.  The  Selected  Reserves  (SELRES),  about  three-fourths 
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of  the  reserves,  serve  1  weekend  a  month  and  2  weeks  a  year  of  active 
duty.  The  remainder  of  the  reserves  are  Training  and  Administrative 
Reserves  (TARs).  TARs  serve  full  time  but  are  attached  to  reserve 
units. 

For  the  purposes  of  this  study,  it  is  assumed  that  TARs  are  equiv¬ 
alent  to  active  duty  personnel  and  that  data  on  SELRES  and  TARs  can  be 
used  to  compare  the  performance  of  reserve  personnel  to  active  duty 
enlisted  men.  In  addition  to  exploring  the  differences  between  these 
two  types  of  reservists  the  study  examines  the  impact  of  training  on  the 
readiness  of  reserve  personnel.  The  extent  of  skill  loss  of  reservists 
and  the  impact  of  training  on  reducing  skill  loss  is  of  great  concern  to 
the  Navy.  This  study  examines  the  effect  of  time  since  active  duty  on 
the  performance  of  personnel.  This  may  be  affected  by  the  amount  of 
time  the  individual  actually  spent  on  active  duty,  so  this  is  considered 
as  well.  Finally,  the  impact  of  the  2  weeks  of  active  duty  for  training 
that  the  reserves  serve  is  analyzed.  This  is  done  by  measuring  the  time 
since  the  last  period  of  active  duty  for  training  for  each  individual. 

The  last  strand  of  research  in  the  study  is  the  assessment  of  the 
importance  of  simulator  training  in  improving  performance.  A  large 
portion  of  this  study  concentrates  on  personnel  and  training  issues 
using  the  simulator  evaluations  as  a  measure  of  performance.  It  is  also 
possible,  however,  to  analyze  the  effectiveness  of  simulators  as 
training  devices.  In  this  context,  the  analysis  measures  the  impact  of 
simulator  training  on  performance  by  measuring  effectiveness  of  person¬ 
nel  as  a  function  of  the  number  of  simulator  trials  they  have  performed 
during  the  study  period.  As  before,  effectiveness  is  measured  by 
simulator  scores.  A  more  extensive  study  might  concentrate  on  the  time 
between  these  trials  as  well  as  a  simple  count  of  their  number,  but  the 
simple  measure  used  here  serves  as  an  initial  attempt  to  analyze  this 
issue. 

The  implicit  assumption  throughout  this  paper  Is  that  performance 
in  a  simulator  is  a  good  proxy  for  performance  in  the  aircraft.  Evi¬ 
dence  on  this  issue  can  be  found  in  earlier  studies  on  the  transferabil¬ 
ity  of  training  from  simulators  to  actual  aircraft  [6,  7].  Previous 
studies  have  found  a  transfer  ratio  of  50  percent  or  more  for  cockpit 
simulators.  The  tasks  being  simulated  in  this  study  do  not  require 
either  visual  or  motion  cues,  hence  increases  in  performance  in  the 
simulator  probably  transfer  at  a  high  rate  to  operational  effectiveness 
in  this  instance. 

Results 


The  factors  that  determine  simulator  scores  were  estimated  using 
regression  analysis.  Simulator  trials  are  scored  on  a  100-point  scale. 
Numerous  individual  tasks  with  different  weights,  are  graded  and  added 
to  determine  an  overall  grade.  Although  the  mean  score  is  close  to  90, 
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there  is  still  variation  across  the  sample.  Regression  results  are 
presented  in  table  1 . 


TABLE  1 

DETERMINANTS  OF  SIMULATOR  SCORES: 
REGRESSION  RESULTS 


Variable 


Coefficient 


( t-statistic) 


Variable  mean 


Constant 

74.5- 

AFQT 

.001 

(0.4) ^ 

69.5 

Paygrade 

0.4 

(1.8K 

5.6 

Sensorl 

-3.1 

(-4.9) 

.34a 

Sensor2 

-2.3 

(-3.7) 

.32a 

Ml 

0.9 

(1.5K 

.37a 

M2 

4.7 

(7.4) 

.28a 

TAR 

-1.5 

(-1.9K 

•  23a 

TSACT 

-.13 

(-2.8) 

10.9 

TSTRA 

-.02 

(-0.3) 

4.5 

TRAMISS 

-0.9 

(-1.2K 

•  21a 

TOTAS 

.25 

(2.8) 

5.7 

TEST 

1.2 

(3.4) 

3.4 

TESTSQ 

-.07 

(-2.1) 

17.1 

YR81 

5.8 

(8.0) 

•  42a 

YR82 

14.6 

(17.8) 

•  38a 

R2  =  .42 

#  Obs  =  1095 

Score  mean  =  87.4,  std.  dev.  =  10.7 


a.  (0,1)  variables.  TAR  =  1  if  TAR,  0  if  SELRES.  TSACT  =  time  since 
leaving  active  duty  (years);  TSTRA  =  time  since  last  2— week  active  duty 
for  training  (months);  TRAMISS  =  1  if  no  record  of  most  recent  training; 
TOTAS  =  total  active  duty  (years);  TEST  =  number  of  recorded  simulator 
trials;  TESTSQ  =  test  squared,  Ml  and  M2  are  variables  representing  the 
type  of  mission  simulated. 


In  general,  all  the  results  are  in  accord  with  their  expected 
signs.  The  magnitude  of  many  of  the  effects  is  much  smaller  than  would 
have  been  expected,  however.  This  is  true,  in  particular,  for  individu¬ 
al  characteristics.  The  variables  measuring  education  have  been  deleted 
entirely  from  the  regression  equation.  The  coefficients,  whether 
measured  in  years  of  education  or  by  degree  status,  were  statistically 
insignificant  and  changed  sign  depending  on  the  specification  of  the 
equation.  AFQT  score  had  a  positive  but  miniscule  impact  on  his 
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simulator  performance.  Individuals  in  the  higher  paygrades  performed 
better,  but  again  the  differences  were  quite  small.  There  is  no  clear 
reason  why  these  factors  have  so  little  explanatory  power.  The  most 
likely  explanation  has  to  do  with  the  nature  of  the  reservists  in  this 
sample.  They  have,  on  average,  15  years  of  experience  between  active 
duty  and  reserve  activity.  The  men  left  in  the  sample  by  that  time  must 
all  be  reasonably  competent  at  their  jobs  to  have  survived  that  long. 
Therefore,  it  is  reasonable  to  expect  that  differences  in  individual 
characteristics  will  be  of  less  importance  than  they  might  have  been 
earlier  in  one's  career. 

The  variables  that  are  included  to  control  for  differences  in 
flight  tasks  all  are  statistically  significant.  These  variables  are 
included  to  control  for  the  possibility  that  individuals  are  sorted  into 
flight  positions  or  crews  into  particular  flights  based  on  their 
personal  characteristics.  These  variables  prevent  any  spurious 
correlation  between  personal  characteristics  and  simulator  scores  based 
on  a  nonrandom  sorting  of  individuals  based  on  the  type  of  task  to  be 
performed. 

The  implications  of  these  results  for  the  readiness  of  reserve 
units  is  very  interesting.  The  most  important  issue  for  reserves  is 
whether  or  not  reserve  crewmen  can  perform  adequately.  The  answer 
obtained  from  this  study  is  a  strong  affirmative.  These  results  indi¬ 
cate  that  SELRES  crewmen  are,  in  fact,  slightly  better  than  their  TAR 
counterparts.  The  difference  is  so  small,  even  though  it  is  statisti¬ 
cally  significant,  that  for  all  intents  and  purposes  SELRES  and  TARs  can 
be  considered  equivalent.  This  result  is  important  because  it  is  often 
assumed  that  SELRES  are  not  as  effective  as  full-time  enlisted  men. 


These  results  do  imply  some  degree  of  skill  loss  over  time,  as 
measured  by  the  negative  coefficient  on  years  since  leaving  active  duty. 
Even  though  this  variable  is  negative  and  statistically  significant,  its 
magnitude  is  quite  small.  For  the  average  reservist,  who  has  been  off 
active  duty  for  11  years,  his  score  is  only  about  a  point  and  a  half 
lower  than  someone  coming  right  from  the  active  force.  This  finding 
suggests  that  the  training  of  reservists,  at  least  for  the  patrol  air¬ 
craft  squadrons,  is  sufficient  to  maintain  their  competence. 

Two  variables  are  included  to  examine  skill  training  more  specifi¬ 
cally.  Time  since  the  last  period  of  active  duty  for  training  has 
virtually  no  impact  on  the  simulator  score.  There  are  two  possible 
explanations  for  these  findings.  The  first  is  that  training  has  little 
impact  on  performance  for  experienced  personnel.  A  more  likely  explana¬ 
tion,  however,  is  that  most  reservists  train  on  a  regular  basis  and  that 
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Che  small  differences  in  time  since  training  are  too  small  to  have  any 
measurable  impact. 

No  record  of  the  last  training  period  was  available  for  one-fifth 
of  the  individuals.  This  may  be  due  simply  to  bad  record  keeping,  but 
it  may  also  indicate  that  no  training  has  been  conducted  for  a  long 
period.  A  variable  to  identify  individuals  with  missing  training 
records  found  that  they  had  slightly  lower  scores. 

Simulator  Training 

Flight  simulators  serve  two  functions.  They  can  be  used  to  measure 
the  readiness  of  individual  crewmen,  as  they  have  been  in  this  analysis. 
They  also  are  training  devices  that  can  be  used  to  improve  the  perform¬ 
ance  of  these  individuals.  Their  usefulness  as  training  devices  can  be 
measured  by  examining  the  impact  of  a  session  in  the  flight  simulator  on 
subsequent  simulated  flights.  Figure  1  displays  the  mean  flight  score 
of  all  individuals  by  the  number  of  simulated  flights  during  the 
observed  period.  The  figure  clearly  displays  a  strong  upward  trend. 
These  graphical  results  are  supported  by  the  regression  coefficients  in 
table  1.  The  regression  estimates  predict  an  average  increase  of  more 
than  one  point  for  each  simulator  flight,  although  this  effect  declines 
slightly  with  each  subsequent  flight. 

Flight  simulator  training  leads  to  substantial  improvements  in 
subsequent  simulated  flights.  Whether  or  not  this  translates  into  like 
improvements  in  performance  during  real  flights  remains  untested.  As 
noted  before,  there  were  too  few  observations  on  operational  flights  by 
reserve  units  to  compare  the  relationship  between  performance  in  simu¬ 
lated  and  actual  flights  carefully.  The  flight  simulators  used  in  these 
tests  are  very  realistic,  however,  and  the  units  that  use  them  place 
great  confidence  in  them.  This  suggests  that  time  in  the  simulator  does 
produce  improvements  in  the  operational  performance  of  VP  crewmen.  The 
magnitude  of  this  effect  may  not  exactly  equal  the  estimates  obtained 
here,  in  fact  it  is  probably  smaller,  but  there  is  evidence  that 
simulators  can  be  used  effectively  to  provide  training  to  these  crew¬ 
men. 

GREW  INTERACTIONS 

The  research  presented  has  concentrated  on  the  scores  each  individ¬ 
ual  operator  received.  The  individual's  score  is  also  affected  by  the 
scores  of  the  other  crew  members.  Table  2  presents  the  simple 


1.  The  results  presented  combine  SELRES  and  TARs  in  a  single  sample. 
Separate  estimates  for  each  population,  tables  A-2  and  A-3,  do  not  show 
any  substantial  differences  in  results. 

2.  The  equations  also  indicate  that  those  were  substantial  improvements 
over  time,  independent  of  additional  training. 
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FIG.  1:  SIMULATOR  SCORES  BY  EXPERIENCE 
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correlations  between  the  individual  scores  of  sensors  1,  2,  and  3.  It 
is  readily  apparent  that  the  score  given  to  the  first  two  sensor 
positions  is  almost  identical  on  most  flights.  This  is  not  surprising 
as  these  two  crewmen  work  as  an  integral  team.  The  correlation  between 
the  score  for  sensor  3  and  the  other  two  positions  is  somewhat  less,  but 
still  substantial.  Regression  analysis  indicates  that  a  1-point  decline 
in  the  score  of  sensor  3  leads  to  half-point  decline  for  the  other  two 
positions,  and  the  reverse  is  true  as  well. 


TABLE  2 

CORRELATION  OF 
CREW  MEMBER  SCORES 


Sensor  1 

Sensor  2 

Sensor  3 

Sensor  1 

1.00 

— 

— 

Sensor  2 

.95 

1.00 

— 

Sensor  3 

.61 

.60 

1.00 

One  final  bit  of  evidence  on  the  importance  of  individual  perform¬ 
ance  to  overall  crew  performance  is  obtained  by  relating  the  individual 
scores  to  the  flight  grade  for  the  whole  aircraft.  The  overall  grade 
depends  both  on  the  performance  of  the  enlisted  sensor  operators  and  the 
officers.  As  can  be  seen  in  table  3,  the  overall  flight  grade  does 
depend  on  the  performance  of  the  enlisted  crew  members.  This  is  true  in 
particular  for  sensor  I.  An  increase  in  the  score  of  sensor  1  by  1 
point  increases  the  overall  grade  by  .2  points.  Conversely,  the  overall 
score  has  a  substantial  impact  on  the  individual  score  for  sensor  1,  but 
the  score  for  sensor  3  seems  to  be  independent  of  the  rest  of  the  crew. 

CONCLUDING  COMMENTS 

The  measurement  of  personnel  performance  is  a  difficult  problem. 
This  paper  has  examined  the  usefulness  of  using  simulators  to  measure 
the  readiness  of  personnel.  The  emphasis  is  on  the  impact  of  training 
on  performance.  The  data  come  from  Naval  Reserve  patrol  aircraft  squad¬ 
rons.  Training  is  particularly  important  for  Reserve  units  who  do  not 
serve  on  a  regular  basis. 


1.  See  tables  A-3  and  A-4. 

2.  Grades  for  sensor  1  and  2  are  so  highly  correlated  that  separate 
effects  cannot  be  estimated.  Hence,  only  sensor  1  is  shown. 

3.  The  effects  of  individual  performance  on  the  overall  flight  grade 
and  the  converse  are  shown  in  tables  A-5 ,  A-6,  and  A-7 . 


-9- 


TABLE  3 


CORRELATION  OF 
CREW  AND  INDIVIDUAL  SCORES 


Crew 

Sensor  1 

Sensor  3 

Grew 

1.00 

— 

— 

Sensor  1 

.42 

1.00 

— 

Sensor  3 

.21 

.61 

1.00 

Although  this  paper  cannot  claim  to  be  more  than  an  initial  effort 

I  in  this  area,  the  results  are  encouraging.  Enlisted  reservists  experfr 
ence  very  little  skill  loss  over  time.  The  typical  reservist,  who  has 
been  out  of  active  duty  for  more  than  10  years,  performs  his  job  jJst 
about  as  well  as  someone  coming  directly  off  active  duty.  The  Selected 
Reserve  enlisted  men  score  as  well  on  simulated  flights  as  their  full¬ 
time  counterparts.  Both  of  these  results  suggest  that  reserve  training 
is  very  good. 

This  study  uses  simulator  scores  both  to  measure  performance  and  to 
analyze  the  value  of  simulators  as  training  devices.  The  drama  tic? 
increase  in  observed  flight  scores  with  each  successive  trial  indicates 
that  the  simulator  provides  a  substantial  amount  of  training.  Suffi¬ 
cient  empirical  data  do  not  exist  to  tie  performance  in  the  simulator  to 
success  in  actual  operational  flights.  Solid  empirical  evidence  on  this 
question  remains  a  topic  for  further  research.  Observational  and  anec¬ 
dotal  evidence,  however,  suggest  that  this  link  does  exist. 

The  final  question  in  this  study  concerned  the  interaction  between 
the  performance  of  the  individual  crewmen.  The  results  are  far  from 
conclusive,  but  they  suggest  that  this  may  be  important  for  some  posi¬ 
tions  and  not  for  others.  The  first  two  sensor  positions  seemed  to 
affect  and  be  affected  by  the  performance  of  the  rest  of  the  crew.  The 
other  position,  on  the  other  hand,  appeared  to  be  largely  unaffected  by 
the  rest  of  the  aircrew. 
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APPENDIX  A 


REGRESSION  RESULTS 


TABLE  A— 1 


DETERMINANTS  OF  SIMULATOR  SCORES 
SELRES  REGRESSION  RESULTS 


Variable 

b 

(t) 

X 

Cons  tant 

AFQT 

73.9 

.02 

(1.2) 

72.2 

Paygrade 

0.3 

(1.0) 

5.7 

Sensorl 

-3.7 

(-5.3) 

•  36a 

Sensor2 

-2.5 

(-3.5) 

.28a 

Ml 

0.8 

(1.2) 

.37a 

M2 

4.6 

6.5 

.28a 

TSACT 

-.01 

(-2.1) 

10.9 

TSTRA 

-.04 

(-0.5) 

4.5 

TRAMISS 

-1.1 

(-1.3) 

•  27a 

TOTAS 

0.4 

(2.4) 

3,9 

TEST 

1.0 

(2.5) 

3.5 

TESTSQ 

-.05 

(-1.4) 

17.8 

YR81 

6.7 

(8.2) 

•  42a 

YR82 

15.3 

(16.1) 

.38a 

R2  =  .44 
#  Obs  =  849 

Score  mean  =  88,  std.  dev.  =  10.7 


a.  (0,1)  variables.  TSACT  —  time  since  leaving  active  duty  (years); 
TSTRA  =  time  since  last  2-week  active  duty  for  training  (months); 
TRAMISS  =  1  if  no  record  of  most  recent  training;  TOTAS  =  total  active 
duty  (years);  TEST  =  number  of  recorded  simulator  trials;  TESTSQ  =  test 
squared. 


A— 1 


TABLE  A-2 


DETERMINANTS  OF  SIMULATOR  SCORES 
TAR  REGRESSION  RESULTS 


Variable  b  ( t)  X 


Constant 

AFQT 

68.6 

-.02 

(-0.7) 

60.0 

Paygrade 

1.5 

(1.8) 

5.3 

Sensorl 

-1.3 

(-0.8) 

.27a 

Sensor2 

-1.7 

(-1.3) 

•  46a 

Ml 

1.2 

(0.9) 

.36a 

M2 

5.2 

(3.7) 

.28a 

TOTAS 

0.1 

(0.6) 

9.6 

TEST 

1.5 

(2.1) 

3.1 

TESTSQ 

-0.1 

(-1.6) 

14.4 

YR81 

3.7 

(2.3) 

.42a 

YR82 

12.9 

(7.6) 

.39a 

R2  =  .35 
#  Obs  =  246 
Score  mean  - 

'  \ 

87.0,  std.  dev. 

=  10.5 

a.  (0,1)  variables.  TSACT  -  time  since  leaving  active  duty  (years); 
TSTRA  =  time  since  last  2-week  active  duty  for  training  (months); 
TRAMISS  =  1  if  no  record  of  most  recent  training;  TOTAS  =  total  active 
duty  (years);  TEST  =  number  of  recorded  simulator  trials;  TESTSQ  =  test 
squared. 
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TABLE  A-3 


REGRESSION  RESULTS:  SENSOR  1  SCORES 


Variable 

b 

(t) 

Constant 

28.4 

Paygrade 

0.5 

(1.6) 

TAR 

0.5 

(0.4) 

Ml 

3.0 

(2.6) 

M2 

1.2 

(1.0) 

Sensor  3 

.60 

(15.2) 

R2  =  .38 
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TABLE  A-4 


REGRESSION  RESULTS:  SENSOR  3  SCORE 


Variable 

b 

(t) 

Constant 

36.2 

Paygrade 

.04 

(0.1) 

TAR 

0.2 

(0.2) 

Ml 

-3.8 

(-3.3) 

M2 

1.3 

(1.0) 

Sensor  1 

.62 

(15.1) 

R2  =  .40 

A-4 


TABLE  A-5 


REGRESSION  RESULTS: 
CREW  SCORE 


Variable 

b 

(o 

Constant 

75.4 

Ml 

1.6 

(2.5) 

M2 

0.6 

(0.9) 

Sensor  1 

.22 

(7.6) 

Sensor  3 

1 

m 

o 

to 

(-0.7) 

R2  =  .19 

Mean  crew  score  =  93.1, 

std.  dev.  =  6.0 
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TABLE  A-6 


REGRESSION  RESULTS:  SENSOR  1  SCORE 

Variable  b  (t) 

Constant  -1.98 

Ml  1.7  (1.5) 

M2  0.6  (0.6) 

Paygrade  .39  (1.5) 

TAR  -0.1  (0.9) 

Crew  Score  .59  (7.5) 

R2  =  .46 
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TABLE  A-7 


REGRESSION  RESULTS:  SENSOR  3  SCORE 


Variable 

b 

(t) 

Cons  tant 

41.0 

Ml 

-3.7 

(-3.2) 

M2 

1.4 

(1.1) 

Paygrade 

.02 

(0.1) 

TAR 

0.3 

(0.8) 

Grew  score 

-.06 

(-0.7) 

R2  =  .40 


A-7 
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